Skip to content

394 multi yoy - allows multiple slope calculations across multi-year periods#464

Closed
cdeline wants to merge 78 commits intoNatLabRockies:developmentfrom
cdeline:394_multi_YoY
Closed

394 multi yoy - allows multiple slope calculations across multi-year periods#464
cdeline wants to merge 78 commits intoNatLabRockies:developmentfrom
cdeline:394_multi_YoY

Conversation

@cdeline
Copy link
Collaborator

@cdeline cdeline commented Aug 15, 2025

This PR builds on #456 and #460 to allow 'multi-YoY' degradation calculations similar to Hugo Quest et al 2023. Fixes #394.
It is enabled by calling degradation.degradation_year_on_year('multi_yoy'=True), or in TrendAnalysis with ta.sensor_analysis(analyses=['yoy_degradation'], yoy_kwargs={'multi_yoy':True, 'label':'center'}) . At the moment there are no example notebooks that demonstrate the process, but these could be developed.

  • Code changes are covered by tests
  • Code changes have been evaluated for compatibility/integration with TrendAnalysis
  • New functions added to __init__.py
  • API.rst is up to date, along with other sphinx docs pages
  • Example notebooks are rerun and differences in results scrutinized
  • Updated changelog
  • Add pytests for multi_yoy = True
  • Add pytests for plotting(detailed=True)

mdeceglie and others added 30 commits January 22, 2025 17:42
Bumps [notebook](https://github.com/jupyter/notebook) from 7.2.1 to 7.2.2.
- [Release notes](https://github.com/jupyter/notebook/releases)
- [Changelog](https://github.com/jupyter/notebook/blob/@jupyter-notebook/tree@7.2.2/CHANGELOG.md)
- [Commits](https://github.com/jupyter/notebook/compare/@jupyter-notebook/tree@7.2.1...@jupyter-notebook/tree@7.2.2)

---
updated-dependencies:
- dependency-name: notebook
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
…tebook-7.2.2

Bump notebook from 7.2.1 to 7.2.2 in /docs
Bumps [tornado](https://github.com/tornadoweb/tornado) from 6.4.2 to 6.5.1.
- [Changelog](https://github.com/tornadoweb/tornado/blob/master/docs/releases.rst)
- [Commits](tornadoweb/tornado@v6.4.2...v6.5.1)

---
updated-dependencies:
- dependency-name: tornado
  dependency-version: 6.5.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
statsmodels 0.14.4 is not able to handle the latest scipy.
Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.5 to 3.1.6.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.5...3.1.6)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-version: 3.1.6
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [requests](https://github.com/psf/requests) from 2.32.3 to 2.32.4.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](psf/requests@v2.32.3...v2.32.4)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.32.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.2.2 to 2.5.0.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](urllib3/urllib3@2.2.2...2.5.0)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-version: 2.5.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
@cdeline
Copy link
Collaborator Author

cdeline commented Aug 26, 2025

Confirmed that TrendAnalysis is compatible with the multi-YoY changes by passing kwargs:
ta.sensor_analysis(analyses=['yoy_degradation'], yoy_kwargs={'multi_yoy':True, 'label':'center'})

@cdeline cdeline changed the base branch from master to development September 16, 2025 21:48
@martin-springer
Copy link
Collaborator

Closing - replaced with #470

martin-springer added a commit that referenced this pull request Mar 5, 2026
* add keyword 'label' to degradation_timeseries_plot, enabling 'left' and 'center' labeling options.

* Update changelog, add pytests, update sphinx documentation

* fix flake8 grumbles

* update pytests to include axes limits

* fix flake8 grumbles

* add 'label' input option to `degradation_year_on_year`. Fixes #459

* add pytests and update changelog.

* flake8 grumbles

* Minor updates to setup.py (constrain scipy<1.16) and refactor degradation_test

* Custom fix for Pandas < 2.0.0 which can't average two columns of timestamps.

* flake8 grumbles

* keep TZ-aware timestamps.  Update pytests to specifically test _avg_timestamp_old_Pandas

* flake8 grumbles

* try to UTC localize the pytest...

* Add .asfreq() to get pytests to agree

* switch to calendar.timegm to hopefully remove TZ issues..

* regardless of uncertainty_method, return calc_info{'YoY_values')

* update _right dt labels to correct _left labels in degradation_year_on_year

* update _avg_timestamp_old_Pandas to allow for numeric index instead of timestamp

* add left label option to degradation_year_on_year

* update degradation_year_on_year, index set to either left, center or right. Consistent with #394 - multi_yoy

* update return for default = none uncertainty option

* degradation_year_on_year - go back to single return when uncertainty_value = None to avoid breaking pytests.

* add multi-year aggregation of slopes in degradation_year_on_year

* add multi_yoy kwarg in degradation_year_on_year to toggle the multi-YoY function.

* update plotting for detailed=True, allow usage_of_points > 2

* flake8 grumbles

* update plotting detailed=True for  (even) and (odd) number of points coloring

* To allow multi_yoy=True in plotting.degradation_timeseries_plot, resample.mean() the YoY_values.

* flake8 grumbles

* Add warning to degradation_timeseries_plot when multi_YoY=True

* update to warning message in plotting.degradation_timeseries_plot

* fix flake8 grumbles

* nbval fixes from qnguyen345-bare_except_error

* Add pandas 3.0 futurewarning handling

* Try again to solve pandas3.0 futurewarning

* attempt 3 to fix nbval

* Add infer_objects to remove futurewarning

* minor inline comment update

* update plotting tests to be relative value, update ordering of module import in plotting.py, per Copilot review.

* update inline comments and whatsnew docs

* Clean up inline comments per Copilot review

* added multi-YoY pytest - still need to catch warnings

* Add a warnings.catch_warnings to the plotting pytest

* flake8 grumbles

* add multi-YoY=True pytest

* updated changelog

* update changelog

* implement copilot suggestions

* linting

* use s instead of ns for pandas 3 compatibility

* update pandas version comparison

* set matplotlib non-gui backend for tests

* set dtype resolution based on pandas version

* linting import order

* boost degradation.py test coverage

* update changelog

* exclude coverage reports with suffixes

* Update rdtools/degradation.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* remove label=None handling, rely on default 'right' behavior

* refactor dt_center  tz handling for old pandas

* simplify _avg_timestamp_old_Pandas

* degradation_timeseries_plot: change rolling median min_periods to rolling_days / 4.

* remove degradation_timeseries_plot(label=) and just default to center=True

* update sensor_analysis() and clearsky_analysis() docstrings to discuss passing `label=right` kwargs

* flake8 updates

* Initial commit - multi-YoY notebook

* pretty-print notebook dataframes with tabulate.

* update notebook requirements to silence pandas warnings

* add multi-yoy nb to tutorials

* fix nblink path

* Change the yoy_values index to be named 'dt'.  Add new illustrations at the end of the multi-YoY notebook.

---------

Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com>
Co-authored-by: martin-springer <martinspringer.ms@gmail.com>
Co-authored-by: Martin Springer <97482055+martin-springer@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants